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Abstract 

Our current health research enterprise is painstakingly slow and cumbersome, and its results seldom translate into 
practice. The slow pace of health research contributes to findings that are less relevant and potentially even 
obsolete. To produce more rapid, responsive, and relevant research, we propose approaches that increase relevance 
via greater stakeholder involvement, speed research via innovative designs, streamline review processes, and create 
and/or better leverage research infrastructure. Broad stakeholder input integrated throughout the research process 
can both increase relevance and facilitate study procedures. More flexible and rapid research designs should be 
considered before defaulting to the traditional two-arm randomized controlled trial (RCT), but even traditional RCTs 
can be designed for more rapid findings. Review processes for grant applications, IRB protocols, and manuscript 
submissions can be better streamlined to minimize delays. Research infrastructures such as rapid learning systems 
and other health information technologies can be leveraged to rapidly evaluate new and existing treatments, and 
alleviate the extensive recruitment delays common in traditional research. These and other approaches are feasible 
but require a culture shift among the research community to value not only methodological rigor, but also the 
pace and relevance of research. 

Keywords: Research methods, Rapid learning, Research review, Stakeholder engagement, Rapid research, Research 
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Background 

Despite increasing demands to produce timely and rele- 
vant research findings, our traditional research process re- 
mains painstakingly slow. Randomized efficacy trials take 
approximately 5.5 years from the initiation of enrollment 
to publication [1], and 7 years or longer after adding the 
time from grant application submission to enrollment ini- 
tiation. Extensive follow-up periods for relevant outcomes 
such as morbidity/mortality as well as delays in participant 
recruitment and publication can extend this time period 
to a decade or longer. During this period, scientific and 
technological advances will occur that may make the 
eventual findings less relevant or even obsolete. For illus- 
tration, Figure 1 shows a few of the salient consumer tech- 
nologies introduced during a typical seven year clinical 
trial. These recent advances in consumer technologies are 
most impactful for mobile and wireless health research, 
[2] but many less mainstream scientific and medical 
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technological advances also occur while clinical trials are 
being conducted. For example, one explanation for the re- 
cently reported negative results of the SAMMPRIS trial 
was that stent technologies and surgery procedures had 
advanced substantially since study initiation [3,4]. 

This protracted period from concept to publication is 
further exacerbated by the slow and limited uptake of re- 
search findings into practice. Balas and Boren have esti- 
mated that it takes approximately 17 years from concept 
to evidence implementation for the 14% of evidence that 
progresses to implementation [5]. A recent report from 
the Presidents Council of Advisors on Science and Tech- 
nology (PCAST) estimated that 3,000 treatments are in 
development and concluded that major upgrades are 
needed in the research system to evaluate these treatments 
[6] . An Institute of Medicine report on clinical trials states 
that "recognition is growing that the clinical trials enter- 
prise in the United States faces substantial challenges im- 
peding the efficient and effective conduct of clinical 
research to support the development of new medicines 
and evaluate existing therapies [7]." Clearly, our current 
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Figure 1 Consumer technology advances missed during a typical RCT published in 2012. 



research enterprise is too slow, inefficient, and cumber- 
some to meet the rapidly evolving demand. 

What are needed are "rapid-learning research systems" 
that integrate researchers, funders, health systems, practi- 
tioners, and community partners asking clinically relevant 
questions, using efficient and innovative research designs, 
and leveraging rich, longitudinal data sets from millions of 
patients. To begin progress toward such a system, we con- 
sidered and have described approaches to make research 
more rapid, responsive and relevant (R3), organized into 
four sections for the purpose of this paper: 1) stakeholder 
engagement, 2) design, 3) review, and 4) infrastructure 



(see Table 1). These suggested approaches are viewed as a 
starting point for a dialogue among the health research 
community to challenge our current cumbersome research 
enterprise and to consider these or other approaches to 
maintain scientific rigor while speeding the process by 
which more responsive and relevant research findings are 
produced. 

Relevance and stakeholder engagement 

Broad stakeholder engagement involving patients, pro- 
viders, health plans, policy makers and other relevant stake- 
holders may seem counterintuitive as a strategy to speed 



Table 1 Issues in promoting rapid research by stage 

Concept through trial preparation 



of research 

Recruitment through follow-up 



Analysis through publication 



Stakeholder 
Relevance 



Design Issues 



• Engage stakeholders via evaluability 
assessment to assist with design of 
practical trials 

• Consider outcomes and measures 
important and relevant to stakeholders 
who will need to act on results 

• Establish stakeholder "citizen-scientist" 
feedback panels; leverage networking 
technologies. 

• Replace the traditional pilot with 
iterative N-of-1 and optimization designs 



Review Issues 



nfrastructure Issues 



• Streamlined grant review process 

• Encourage reviewers to consider 
innovative designs that speed research 

• Streamline IRB approval process, 
especially for low risk studies 

• Use of data standards and common 
data elements to improve research 
efficiency and facilitate data sharing 

• Create rapid learning systems that can 
generate data to test multiple 
competing hypotheses and develop 
predictive models 

• Create national biobank/bio-samples 
systems 



• Ongoing engagement with 
stakeholders on methods to improve 
recruitment and follow-up retention 

• Submit preliminary findings to 
stakeholders for review and direction- 
setting 



• Consider within-subject and MINC to 
typical comparison conditions 

• Leverage technology to automate 
RCTs when possible 

• Consider alternatives to the two-arm 
RCT including factorial, within subject, 
pragmatic, quasi-experimental, and rapid 
learning designs 

• Rapid modification approvals from 
IRBs 



• Use practice network registries to 
speed recruitment, provide enriched 
histories & follow-ons 

• Leverage existing EHR and other rapid 
learning data systems to rapidly test 
hypotheses 



• Submit initial results to stakeholders 
for assistance with interpretation, 
relevance, dissemination and forming 
next study questions 

• Share presentations with stakeholders 
at policy and practice venues 



• Report proximal outcomes while 
follow-up data collection continues 



• Encourage online and open access 
publication 

• Incentives to speed manuscript 
reviews 



• Robust policies and procedures for 
data sharing and merging 

• Improved systems for disseminating 
findings to appropriate stakeholders 
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research, but this time investment has the potential to im- 
prove the recruitment and retention of study participants, 
thus increasing the pace of conducting the study. More im- 
portantly, stakeholder engagement increases the likelihood 
that findings will be relevant to stakeholders and more 
readily adopted into practice, thereby making the overall re- 
search pipeline more efficient. These "evaluability assess- 
ments" [8] or participatory approaches are considered key 
to facilitating the adoption of research findings by practi- 
tioners [9]. The Patient Centered Outcomes Research Insti- 
tute (PCORI), for example, is creating public advisory 
groups and soliciting patient input on specific comparative 
effectiveness questions that are relevant to practitioners and 
stakeholders [10]. 

An ongoing relationship between researchers, healthcare 
providers, health plans, and patients is critical to a better, 
faster research system. Clinical trial recruitment is a major 
problem with about 90% of US trials failing to meet enroll- 
ment goals [7]. The NIH Health Care System Collaboratory 
(HCSC) offers an important resource for rapid research. Like 
the HMO Research Network and the VA QUERI program, 
the HCSC will make available opportunities to conduct 
large-scale studies within well-organized healthcare delivery 
systems [11]. Research embedded in organized delivery sys- 
tems and networks enhances not only research relevance, 
but also facilitates recruitment, retention, study start-up, op- 
erations, data capture, and integration into practice. 

An accelerated research system can also use information 
technologies to speed the process of seeking and obtaining 
stakeholder feedback. Consistent with a citizen-scientist 
model [12], a virtual network of various stakeholders can 
participate and provide feedback throughout the research 
process using online surveys, virtual meetings, and social 
media systems [13]. Via innovative technologies, stake- 
holder feedback can be obtained efficiently to increase re- 
search relevance and responsiveness, even for researchers 
who are not fully integrated into the practice setting or 
community where their research findings are likely to be 
translated into practice. 

Rapid research designs 

Traditional study designs and procedures are well- 
established, rigorous, and notoriously slow and costly. 
This belabored research process typically begins with 
pilot trials that we posit have limited benefit, are used 
inappropriately to estimate effect size [14], and often 
prematurely concretize a less than optimal intervention. 
Instead, we recommend replacing the traditional pilot 
trial with a more flexible iterative intervention testing 
and optimization approach, analogous to the agile soft- 
ware development process that places a premium on 
failing early to succeed later [15]. For example, N-of-1 
trial designs provide intervention development flexibil- 
ity. With the increasing availability of intensive 



longitudinal data from wireless sensors and mobile de- 
vices, N-of-1 trials can be rapidly implemented and 
provide results congruent with a more personalized 
medicine approach [16], and Bayesian analyses from a 
series of such trials [17] may provide sufficient evidence 
of generalizability to limit the need for a larger trial. 

Intervention optimization designs such as fractional fac- 
torial and sequential multiple assignment research trials 
(SMART) are particularly valuable when the intervention 
development questions involve combinations or sequences 
of intervention components [18]. Dynamic system models 
have also been used to optimize treatments [19]. Some 
optimization approaches may take more time than the 
traditional pilot trial, but the pace of the overall research 
enterprise will be improved by more quickly discarding or 
modifying interventions that are unlikely to be found ef- 
fective in larger and more expensive trials. 

Within the traditional RCT, researchers have a number 
of design decisions that can increase efficiency. Trials that 
utilize within-group designs in which participants serve as 
their own controls can speed the research process by redu- 
cing the number of study participants needed to detect 
outcomes, and can often simplify study procedures as well. 
The Minimal Intervention Needed for Change (MINC) 
standard [20] provides a standard pragmatic comparison 
anchor across studies for comparative effectiveness re- 
search. The VA is adopting a "point of care" randomization 
that computer-randomizes patients to different treatments, 
and then uses adaptive algorithms to change allocation of 
new patients as evidence accumulates [21]. Recent techno- 
logical advances make it possible to conduct "automated 
RCTs" in which the enrollment, random assignment, inter- 
vention delivery, and outcome assessments are fully auto- 
mated. To fully realize the potential of automated RCTs 
and other rapid learning systems, the nature of and proce- 
dures for informed consent need to be resolved. 

Follow-up periods also can be shortened or segmented. 
Results can be analyzed at the point where the maximal 
benefit of the intervention is hypothesized to occur. 
Longer-term outcomes can be modeled from these results, 
or one of the investigators can remain blind to conduct the 
follow-up portion of the study and publish the follow-up 
results separately. 

In addition to improving the efficiency of RCTs, we also 
need to consider alternative designs that may be more ap- 
propriate to the research question and provide more rapid 
and relevant answers. A range of within-subject and 
quasi-experimental designs such as interrupted time series 
[22], stepped wedge [23], and regression discontinuity [24] 
may have less internal validity than the RCT, but offer a 
number of advantages [25]. For example, these quasi- 
experimental approaches facilitated participation of the 
major Minnesota health insurers in the DIAMOND de- 
pression treatment program [26]. These designs may be 
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particularly appropriate for evaluating treatments already 
adopted in practice. 

Rapid review processes 

It takes 9 to 11 months from NIH grant submission to 
funding [27]. If revised and resubmitted, and assuming a 
six month revision period, it can take two years from initial 
submission to the award of a revised (Al) NIH grant appli- 
cation. During this time, science and technology continue 
to advance; research partnerships, especially with non- 
research stakeholders, must be maintained; and the re- 
search questions may become less relevant or timely. 

Grant review and funding processes could be stream- 
lined in a number of ways. For the Recovery Act Chal- 
lenge Grants [28], a flexible, two-stage review process 
was implemented to reduce to five months the time from 
receipt to funding. Rapid review processes are already 
used by the NIH for time sensitive natural experiments 
[29,30]. In response to the SARS (Severe Acute Respira- 
tory Syndrome) outbreak, the Canadian Institutes of 
Health Research developed and issued a funding an- 
nouncement that resulted in 18 submissions within 2 
weeks, and these submissions were reviewed and four ap- 
proved for funding within 10 days of submission [31]. 
These rapid review examples clearly indicate that it is 
possible to review and fund research applications quickly 
when necessary, and that such rapid review systems 
should be considered for a broader range of research, in- 
cluding timely and pressing clinical and public health 
questions. 

The grant application review process could facilitate 
more rapid research not only by reviewing more effi- 
ciently, but also by placing a greater premium on more 
rapid and innovative research designs. Despite the 
addition of innovation as an NIH review criteria, a re- 
cent study of grant applications revealed that novelty is 
associated with a 4.5 percentile point drop, and that 
feasibility concerns did not contribute substantially to 
this "novelty penalty" [32]. Innovative designs, especially 
those that speed the pace of research relative to trad- 
itional designs, should be rewarded, not penalized. 

Institutional Review Boards (IRBs) also should consider 
streamlining review procedures. Slowness of research 
should be considered a risk, both to study participants 
who may continue in their assigned treatment even as 
newer treatments become available, and to the broader 
public who are delayed getting answers to relevant re- 
search questions. Revisions to the Common Rule are an- 
ticipated to allow for a more flexible and rapid review 
process [33]. 

Online and open access publication practices have greatly 
reduced the time from acceptance to publication [34] but 
could be further facilitated by a better or more incentivized 
process for acquiring reviewers and obtaining reviews. As 



Green noted, new technologies for publication, systematic 
reviews, and dissemination of evidence-based guidelines re- 
duce the time from research findings to practitioner adop- 
tion, but the publication and dissemination process should 
continue to be reviewed to further reduce the time lapses 
between the various stages of the dissemination and imple- 
mentation process [9]. 

Infrastructure for rapid research 

Improving our research infrastructure has the potential not 
only to speed the pace of research, but also increase its 
rigor and relevance. The health system has lagged decades 
behind other sectors in IT implementation [35] . As a result, 
health research has been severely constrained by a data- 
poor environment in which acquiring needed research data 
is expensive, difficult, and time-consuming. Since the rapid- 
learning health system and learning healthcare system con- 
cepts were advanced in 2007 [36], major investments have 
been made in databases and learning networks to take ad- 
vantage of the research potential of electronic health re- 
cords. It is now possible to conduct some studies in weeks 
or months instead of years. The FDA mini-Sentinel system 
accesses 125 million patient records to generate several 
studies per week on drug safety questions [37]. Large 
biobanks are now coming on-line at Kaiser-Permanente, 
[38] the Veterans Health Administration [39], the EN- 
CODE network, [40], and the UK Biobank [41]. Using these 
and other patient databases, researchers have been able to 
assess the unintended effects of treatments [42] and pro- 
duce outcome findings comparable to RCTs [43]. 

Large future investments are now being considered that 
could offer extraordinary opportunities for researchers 
and a faster, more efficient infrastructure for rapid learn- 
ing research. The NIH Director has proposed a new na- 
tional patient-oriented research system with electronic 
health records databases, including genomics, for 20-30 
million patients [44], and PCORI recently released a 
funding announcement to support development of the 
National Patient Centered Clinical Research Network [10]. 
The Big Data to Knowledge (BD2K) initiative [45] also 
provides the opportunity to leverage these large data sets 
for rapid research. 

Researchers can accelerate the collective pace of learn- 
ing with greater attention to reporting comparable data. 
There have been a number of efforts to encourage the use 
of common data elements [46]. Standardized outcome 
data are particularly problematic for patient-reported out- 
comes. To address this problem, there have been consen- 
sus measurement efforts such as PhenX [47] as well as 
efforts to co-calibrate various patient-reported outcome 
measures on a single metric [48] . 

The National Research Council report on Precision 
Medicine calls for a new science commons and national 
learning system that will revolutionize biomedical research, 
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clinical care, and public health [49]. One of the benefits of 
such a system is that researchers can more readily target 
drug approval studies to predicted high-response popula- 
tions, and cut years from the drug research process. 
Gleevec, an anti-cancer drug, was approved in a trial of 
only 54 patients because nearly all showed benefit [6]. 
With targeted therapeutics and research, it took only four 
years from target discovery to drug approval for the lung 
cancer drug Xalkori [50]. Rapid learning systems of large 
patient populations appear to provide the infrastructure to 
rapidly evaluate treatments. 

Conclusions 

We have outlined a number of actions to enhance rele- 
vance, streamline design, speed review, and use new re- 
search infrastructures to make research more rapid, 
relevant, and responsive to the 21 st century demands on 
health research. This transformation to a rapid research 
learning system will require a concerted effort by research 
funders, academic institutions, healthcare systems, re- 
searchers, and a variety of practice, community, and policy 
stakeholders to produce a culture change among the 
health research community. Will we continue to use lim- 
ited funds to support the currently slow and cumber- 
some research enterprise that produces costly results that 
may not be relevant or easily translated into practice, or 
are we willing to pursue alternative approaches that in 
other research disciplines have produced rapid and rele- 
vant improvements? 

The rationale and opportunities for such a culture 
change have never been greater or rapid answers more 
needed. Our recommendations to speed research are un- 
doubtedly incomplete, and we invite the research commu- 
nity to contribute additional recommendations to increase 
the speed and relevance of the research enterprise. This 
call to streamline and speed the research process is also 
likely to be met with skepticism, especially among those 
who fear that methodological rigor might be compromised 
in a quest for greater efficiency. We believe that the rec- 
ommendations outlined in this paper can be achieved 
without compromising scientific rigor, and any efforts to 
streamline research should be judged based on methodo- 
logical soundness. We are convinced, however, that the 
currently dominant health research paradigm is too slow 
and inefficient to address today's challenges, and that we 
must produce a more rapid, responsive, and relevant re- 
search enterprise. 
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